Skip to content

fix: OOM on cold parse (V8 SlicedString retention) + kiro projectPath for repo attribution - #984

Open
Enclavet wants to merge 2 commits into
getagentseal:mainfrom
Enclavet:fix/kiro-parse-oom
Open

fix: OOM on cold parse (V8 SlicedString retention) + kiro projectPath for repo attribution#984
Enclavet wants to merge 2 commits into
getagentseal:mainfrom
Enclavet:fix/kiro-parse-oom

Conversation

@Enclavet

Copy link
Copy Markdown

Summary

Two related fixes for the kiro provider and shared parser, ordered deliberately (see "Landing order" below):

  1. fix(parser): cold-parse OOM caused by V8 SlicedString retention in the session cache
  2. fix(kiro): emit the session's full working directory as projectPath so codeburn yield and sync push --attribution (feat(sync): opt-in git attribution spans on sync push --attribution #848) can resolve kiro sessions to their git repos

1. The OOM

codeburn sync push --since=all (and any cold parse of a large kiro store) crashed with
FATAL ERROR: Ineffective mark-compacts near heap limit at V8's default ~4GB heap.

Root cause: String.prototype.slice returns a V8 SlicedString — a view that retains a
reference to its entire parent string. The parsers store short previews of message text
(userMessage.slice(0, 500/2000)) in the long-lived session cache. Kiro CLI session entries
routinely carry 100KB+ strings (agent-injected system prompts, tool results), so every cached
preview pinned its full parent buffer for the life of the process.

Measured on 3.2GB of kiro CLI session files (6,659 files, largest 40MB):

Scenario Peak RSS
Cold parse, default heap, before 4.33GB → OOM
Cold parse, 8GB heap, before (kiro alone) 5.67GB
After flatSlice at kiro capture sites 0.91GB
After shared cache-build sites too 0.64GB
Original failing command, cold, default heap, after 0.89GB — completes

Warm runs were always fine (~0.29GB) because the cache's JSON round-trip flattens strings on
load — which made the bug look intermittent: it only fired on a cold or invalidated cache.

Fix: flatSlice(s, max) (bounded prefix as a flat copy via Buffer round-trip) and
flatString(s) (unconditional flat copy, for short strings that are views — regex match
groups and trim() results). Applied at the six kiro userMessage capture sites, the three
shared cache-build sites in parser.ts (protects all providers), the title/agentType
trim().slice() sites, and extractToolNames (regex match groups retain the entire scanned
content buffer).

2. kiro projectPath

The kiro provider read the full working directory from session metadata (meta.cwd for CLI
sessions, workspacePaths[0] for v2 IDE sessions, workspaceDirectory for workspace
sessions) but discarded it via basename(), keeping only the leaf name for display. Repo
grouping in yield.ts therefore never resolved kiro sessions to a git repo — codeburn yield
misgrouped them under the cwd fallback, and sync push --attribution produced 0 facts for all
kiro sessions. On my machine this fix took a 6-month attribution window from 59 to 318 facts.

Behavior changes to be aware of:

  • kiro calls now flow through worktree canonicalization: sessions in linked git worktrees
    report under the main repository's project name (consistent with claude/codex)
  • workingDirectory is now populated on kiro calls
  • PROVIDER_PARSE_VERSIONS.kiro is bumped (project-path-v1): cached entries predate
    projectPath and are served without re-invoking the parser, so without the bump this fix
    silently no-ops for every warm cache. Upgrading triggers a one-time cold kiro re-parse.
  • resolveCanonicalProjectPath is now memoized on cwd (kiro's per-call canonicalization
    measured ~+5% cold parse before memoization; the same cache benefits all providers)

Landing order (important)

The version bump forces a cold kiro re-parse on upgrade — exactly the workload that OOM'd
before commit 1
. These commits must land together, in this order. Please don't split or
reorder them.

Tests

  • tests/flat-slice.test.ts: prefix/identity/multi-byte behavior, a pinned decision for
    mid-surrogate-pair cuts (U+FFFD), and heap-growth property tests proving no parent retention
    for both flatSlice and regex-match flatString
  • tests/kiro-projectpath.test.ts: projectPath emission fixtures for all three kiro session
    formats (CLI, v2 IDE, workspace-session), a fingerprint-change assertion, and a regression
    test seeding a pre-bump cache entry and proving the re-parse recovers projectPath
  • tsc clean; kiro/sync/yield/parser suites pass (178 tests)

… cache

String.prototype.slice returns a V8 SlicedString: a view that retains a
reference to its ENTIRE parent string. The parsers store short previews
of message text (userMessage.slice(0, 500/2000)) in the long-lived
session cache. Session files routinely carry 100KB+ strings (agent-
injected system prompts, tool results), so every cached preview pinned
its full parent buffer for the life of the process.

Measured on 3.2GB of kiro CLI session files (6,659 files, largest 40MB):

  cold parse, default heap, before:  4.33GB peak -> OOM crash
  cold parse, 8GB heap, before:      5.67GB peak (kiro provider alone)
  after kiro flatSlice:              0.91GB peak
  after parser.ts cache sites too:   0.64GB peak
  original failing command (cold,
  default heap, all providers):      0.89GB peak -> completes

Warm runs were always fine (~0.29GB) because the cache's JSON round-trip
flattens the strings on load — which made this bug appear intermittent:
it only fired on a cold or invalidated cache.

Fix: flatSlice() in content-utils.ts forces a flat copy via Buffer
round-trip. Applied at the six kiro userMessage capture sites and the
three shared cache-building sites in parser.ts (protects all providers).

Regression test asserts the no-retention property via bounded heap
growth over 1000 large-parent slices.

AI-Origin: human
The kiro provider was reading the full working directory from session
metadata (meta.cwd for CLI sessions, workspacePaths[0] for v2 IDE
sessions, workspaceDirectory for workspace sessions) but discarding it
via basename(), keeping only the leaf name for display. This meant
computeAttributionRecords could never resolve kiro sessions to a git
repo, so `codeburn sync push --attribution` produced 0 facts for all
kiro-originated sessions.

Now passes the full path as projectPath on emitted ParsedProviderCalls,
which buildRepoGroups uses to resolve git identity and correlate
commits with sessions via timestamp windows. The path stays local:
only the normalized origin remote egresses in attribution spans.

Behavior changes beyond attribution:
- kiro calls now flow through canonicalizeProviderCallProject, so kiro
  sessions in LINKED GIT WORKTREES canonicalize to the main repository:
  their report project name changes from the worktree dir name to the
  main repo name (consistent with claude/codex behavior).
- workingDirectory is now populated on kiro calls.
- PROVIDER_PARSE_VERSIONS.kiro bumped (project-path-v1): cached entries
  predate projectPath and are served without re-invoking the parser, so
  without the bump this fix silently no-ops for every warm cache. The
  bump forces a one-time cold kiro re-parse on upgrade.

ORDERING: this commit must land WITH (or after) the preceding
SlicedString OOM fix. The forced cold re-parse it triggers is exactly
the workload that OOM'd before that fix on multi-GB kiro stores.

Perf: per-call canonicalization added a measured +5% to cold parse
(.git-marker lstat walk per call). resolveCanonicalProjectPath is now
memoized on cwd (cleared with the session cache), removing the
redundant walks for all providers.

Tests: projectPath emission fixtures for all three session formats
(CLI, v2 IDE, workspace-session), fingerprint-change assertion, and a
regression test seeding a pre-bump cache entry and proving the re-parse
recovers projectPath.

AI-Origin: human
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants